The present invention relates to the field of high resolution hyperspectral imaging.
Hyperspectral imaging systems in general are known, and have been used for a diverse range of remote sensing and other analytical techniques, such as is disclosed, for example, in U.S. Pat. No. 5,790,188 and the related U.S. Pat. No. 6,211,906. Hyperspectral imaging has also been used in conjunction with microscopic optical systems, such as disclosed, for example, in U.S. Pat. No. 6,495,818. In such systems, radiation reflected by or emanating from a target or specimen is detected in a large number of narrow contiguous spectral bands, producing a data set which is distributed not only spatially, but spectrally as well. That is, for each pixel within an image of the target, information is recorded in each of the spectral bands, thereby producing a three-dimensional hyperspectral image cube, in which spectral information for each pixel is distributed across a spectral axis perpendicular to the spatial axes.
Previously, hyperspectral imaging has been hampered by difficulties in accurately aiming, imaging, processing and displaying data in real-time. One such problem in particular is that previously known hyperspectral imaging systems lack the ability to quickly and accurately adapt their imaging devices to the size and relative position of the imaging target, and to process and present spectral information in a timely manner to provide real-time user review and refinement of image acquisition.
For example, U.S. Pat. No. 5,608,520 describes an apparatus for application of laser light to a target such as a tumor, where light reflected from the laser is monitored by a spectrometer over a fiber optic line. A separate video camera provides an image of the tumor to permit visual monitoring of the procedure. The imaging devices are operated completely independently of one another. That is, there is no provision for coordinated processing of both the video and spectrometer image data to provide real-time, integrated display of the video and spectral information.
Another example is found in U.S. Pat. No. 6,529,769 B2, in which an endoscope is coupled to a spectrometer and a video camera (a “CCD” camera employing a charge-coupled device for its imaging sensor). Images from each are separately displayed side-by-side on a display unit. There is no provision for adjustment of the endoscope's imaging elements (e.g., the fiber optic elements at the probe-end of the endoscope) to align the images received by the video camera and by the spectrometer relative to one another, nor any provision for processing to identify and optimize the imaging of a target area.
Similarly, U.S. Pat. No. 6,667,761 discloses an “instrument visualization system,” in which a video camera and a sensor, such as a pyrometer, are pre-aligned on a sensor platform, so that the sensor's detection element is aligned with a spot within the video camera's field of view that is represented by a targeting artifact on a video display. This system has no provision for altering the spatial relationship between the sensor and the target (such as by moving closer or farther away from the target), or any means for processing the spectral information obtained across the entire field of view of the spectrometer to extract desired spectral reflectance information. The latter concern is an important factor in optimizing spectral imaging performance, where the target image information might be substantially diluted if a large fraction of background area is in the sensor's field of view.
Thus, there is a need for an improved system and method for real-time, simultaneous image data acquisition, processing and display from non-visual and visual imaging devices, including obtaining improved sensor performance from enhanced real-time sensor positioning, such as by better matching the sensor's field of view to the size of a target.
The present invention addresses the foregoing problems by providing an integrated system for spectral imaging of a target. The system includes a spectrometer imaging device (such as a fiber optic camera lens communicating with a spectrometer), a video imaging device (such as a CCD camera), and a distance sensor, all mounted in an integrated sensor probe. The video imaging device and the spectrometer imaging device are arranged on the integrated sensor probe such that their respective fields of view at least partially overlap. The integrated sensor probe communicates with a computer which processes the image data received from the imaging devices and the distance sensor.
The computer displays on a display unit an image obtained from the video imaging device. Overlaid on the video image is a computer-generated graphic symbol representing the shape and location of the spectrometer imaging device's “ground instantaneous field of view” to provide an operator with a real-time indication as to which portion of the video camera's field of view is being imaged by the spectrometer. The ground instantaneous field of view (“GIFOV”) is the extent of the two-dimensional area within view of the device at a given distance from the spectrometer imaging device. For example, a spectrometer imaging device with a 30 degree-wide cone-shaped field of view, located 100 cm from a target, will have a GIFOV in the shape of a circle with a radius of 25.88 cm.
In the present invention, the GIFOV of the spectrometer imaging device at any particular instant is determined, in real-time, by the computer. The computer determines the GIFOV from the instantaneous distance of the spectrometer imaging device to a target sensed by the distance sensor, coupled with a predetermined actual field of view profile. The actual shape of the spectrometer imaging device's field of view is predetermined and stored in a profile to improve imaging accuracy, recognizing that the imaging device's filed of view may not be precisely conical.
In addition to display of the video image and the overlaid spectrometer GIFOV, the computer may also be programmed to display (in an adjacent area of the display screen, for example), a real-time representation of data obtained from the GIFOV spectrometer. An operator would thereby be provided with real-time information regarding the spectral emissions from the target, while simultaneously being able to visually correlate the spectral information with the specific target area being imaged by the spectrometer.
It is a further objective of the present invention to enhance the accuracy and speed of image data collection and processing by automating various aspects of imaging target acquisition and optimization. A computer-assisted calibration procedure may be used with the present invention to provide an initial definition of the actual extent of the field of view of the spectrometer imaging device. In this procedure, the computer displays a “virtual grid” pattern of known dimensions (for example, a 100×100 grid) on the display while the video imaging device and the spectrometer imaging device are viewing a simple target, such as a black target background. A spectrally-contrasting object, such as a small white target, is sequentially moved across each of the cells in the virtual grid, and the response of the spectrometer obtained at each cell position. This procedure will allow rapid identification of the actual limits of the field of view of the spectrometer imaging device. Further, by knowing both the distance from the imaging device to the target background and the actual two-dimensional extent of the GIFOV at the target background, the angular limits of the spectrometer imaging device's actual field of view may be derived and stored as the imaging device's actual field of view profile for later retrieval. By storing the actual field of view limits in angular form, the computer processor then needs only a target distance signal from the sensor probe's distance sensor when subsequently viewing a target in order to calculate and display the actual GIFOV of the spectrometer imaging device over the video image display.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying figures.